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TWO-DIMENSIONAL PYRAMID FILTER ARCHITECTURE 

RELATED APPLICATIONS 

This patent application is related to U.S. Patent Application Serial No. 
09/754,684, titled "Multiplierless Pyramid Filter," filed January 3, 2001, by Tinku 

Acharya; U.S. Patent Application Serial No. , titled "Two-Dimensional Pyramid 

Filter Architecture," (attorney docket 042390.P1 1275), filed March 26, 2001 , by Tinku 

Acharya; U.S. Patent Application Serial No. , titled "Pyramid Filter," (attorney 

docket 042390.P11211), filed March 28, 2001, by Tinku Acharya; and concurrently 

filed U.S. Patent Application Serial No. , titled "Two-Dimensional Pyramid 

Filter Architecture," (attorney docket 042390.P1 1276), filed March 2001 , by Tinku 
Acharya, all assigned to the assignee of the presently claimed subject matter and 
herein incorporated by reference. 

BACKGROUND 

This disclosure is related to pyramid filters. 

In image processing it is often desirable to decompose an image, such as a 
scanned color image, into two or more separate image representations. For example, a 
color or gray-scale document image can be decomposed into background and 
foreground images for efficient image processing operations, such as enhancement, 
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compression, etc., as are at times applied in a typical photocopying machine or scanner 
device. In this context, this operation is often referred to as a descreening operation. 
This descreening is also sometimes applied to remove halftone patterns that may exist 
in an original scanned image. For example, these halftone patterns may cause 
objectionable artifacts for human eyes if not properly removed. The traditional approach 
for this decomposition or descreening is to filter the color image in order to blur it. 
These blurred results are then used to assist in determining how much to blur and 
sharpen the image in order to produce the decomposition. Typically this blurring can be 
achieved using a "symmetric pyramid" filter. Symmetric pyramid finite impulse 
response (FIR) filters are well-known. 

One disadvantage of this image processing technique, however, is that the 
complexity increases many fold when a number of pyramid filters of different sizes are 
applied in parallel in order to generate multiple blurred images, to apply the technique 
as just described. A brute force approach for this multiple pyramid filtering approach is 
to use multiple FIR filters in parallel, as illustrated in FIG. 1. Such an approach 
demonstrates that the design and implementation of fast "symmetric pyramid filtering" 
architectures to generate different blurred images in parallel from a single source image 
may be desirable. 

The numbers provided in parenthesis for each FIR block in FIG. 1 represents the 
pyramid filter of corresponding length. For example, (1, 2, 1) are the filter coefficients 
for a symmetric pyramid finite impulse response (FIR) filter of order or length 3. 
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Likewise, (1 , 2, 3, 2, 1) are the coefficients for an FIR pyramid filter of order 5, (1 , 2, 3, 
4, 3, 2, 1) are the coefficients for an FIR pyramid filter of order 7, (1, 2, 3, 4, 5, 4, 3, 2, 
1) are the coefficients for an FIR pyramid filter of order 9, and so forth. 

Unfortunately, the approach demonstrated in FIG. 1 has disadvantages. For 
example, inefficiency may result from redundant computations. Likewise, FIR 
implementations frequently employ multiplier circuits. While implementations exist to 
reduce or avoid the use of multipliers, such as with shifting and summing circuitry, that 
may then result in increased clocking and, hence, may reduce circuit through-put. A 
need, therefore, exists for improving pyramid filtering implementations or architectures. 



BRIEF DESCRIPTION OF THE DRAWINGS 

Subject matter is particularly pointed out and distinctly claimed in the concluding 
portion of the specification. The claimed subject matter, however, both as to 
organization and method of operation, together with objects, features, and appendages 
thereof, may best be understood by reference of the following detailed description when 
read with the accompanying drawings in which: 

FIG. 1 is a block diagram illustrating a brute force approach to implementing a 
finite impulse response (FIR) multiple pyramid filtering architecture; 
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FIG. 2 is one embodiment of a one-dimensional multiplierless pyramid filter; 

FIG. 3 is one embodiment of a two-dimensional pyramid filter architecture; 

FIG. 4 is a table/matrix showing an example of a matrix that may result from 
implementing a two-dimensional pyramid filter architecture, such as one that may be 
implemented by the embodiment of FIG. 3; 

FIG. 5 is a table/matrix showing an example of a two-dimensional signal that 
may be operated upon by a two-dimensional pyramid filter architecture; 

FIG. 6 is a table/matrix showing an example of applying a one-dimensional 
pyramid filter kernel both row-wise and column-wise; 

FIG. 7 is the table/matrix of FIG. 6 for k=9; 

FIG. 8 is a table/matrix showing the result of applying a one-dimensional pyramid 
filter to the rows of a two-dimensional input signal sample matrix; and 

FIG. 9 is a table/matrix showing the result of applying a one-dimensional pyramid 
filter to the columns of a two-dimensional input signal sample matrix. 
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DETAILED DESCRIPTION 

In the following detailed description, numerous specific details are set forth in 
order to provide a thorough understanding of the claimed subject matter. However, it 
will be understood by those skilled in the art that the claimed subject matter may be 
practiced without these specific details. In other instances, well-known methods, 
procedures, components and circuits have not been described in detail in order so as 
not to obscure the claimed subject matter. 

As previously described, pyramid filtering, in particular, symmetric pyramid 
filtering, may be employed in connection with color images or color image processing in 
order to decompose or descreen the image, such as into a background and foreground 
image, for example. Although the claimed subject matter is not limited in scope in this 
respect, in such a context, pyramid filtering architectures that reduce computational 
complexity or processing and/or hardware cost are particularly desirable. Likewise, 
implementations that are multiplerless, that is do not specifically employ multiplication in 
the implementation, are also desirable usually because such implementations or 
embodiments are cheaper to implement than those that employ or include multiplier 
circuits. 

Although the claimed scope is not limited in scope in this respect, FIG. 2 
illustrates an embodiment 200 of a one-dimensional pyramid filter, such as described in 
more detail in aforementioned U.S. Patent Application Serial No. 09/754,684, titled 
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"Multiplierless Pyramid Filter/' by T. Acharya (attorney docket no. 042390.P10722), filed 
on January 3, 2001. Embodiment 200 comprises a unified multiplierless cascaded 
symmetric pyramid filtering architecture to generate a multiple number of filtered output 
signal streams for a series or sequence of pyramid filters having different orders, the 
generation of the output signal streams occurring in parallel. In this particular 
embodiment although, again, the claimed subject matter is not limited in scope in this 
respect, a filtered output signal stream is produced on every clock cycle for each 
pyramid filter of a different order being implemented. Therefore, in addition to being 
computationally efficient, this particular embodiment produces good results in terms of 
throughput. However, as previously indicated, this particular embodiment implements a 
one-dimensional pyramid filter. 

FIG. 2 is understood in the context of specific notation. For example, an input 
source signal, X, may be designated as follows: 

X = (Xo, Xi, , X,_2 ? Xm, Xj, Xj+1, Xj+2, .. .) 

In digital or discrete signal processing, filtering may be expressed as a 
convolution, ® , of the input signal, X, and a filter, F, in this context a digital filter of finite 
length, referred to here as a finite impulse response (FIR) filter. Therefore, the filtered 
output signal stream is indicated as follows: 
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Y = X ® F 



As previously described, the particular embodiment in FIG. 2 employs pyramid 
filters. These filters are typically implemented using digital filters of lengths or orders 
that are odd, such as 3, 5, 7, 9, etc. Odd numbers or orders, in this context, may be 
expressed in the form 2N-1 , where N is a positive integer greater than two, for example. 
Some examples of such digital filters are as follows: 



F 3 = (1,2,1) 

F 5 =(1,2, 3, 2,1) 

F 7 = (1,2, 3, 4, 3, 2, 1) 

F 9 = (1,2, 3, 4, 5, 4, 3, 2,1) 



F M = (1, 2, 3, ... , N, ... , 3, 2, 1) (where, in this context, M = 2N-1) 

For the foregoing filters, the filtered output signals or output signal streams may 
be represented as follows: 

B 3 = X ® F 3 = (b 0 3 , bi 3 , ... , b M 3 , b 3 , b i+1 3 , ....) result of input signal X filtered 

by F 3 
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B 5 = X ® F 5 = (b 0 5 , bi 5 , ... , bi-i 5 , bj 5 , b i+ i 5 , ....) result of input signal X filtered 

by F 5 

B 7 = X ® F 7 = (b 0 7 , b/, ... , b M 7 , bj 7 , b i+ i 7 , ....) result of input signal X filtered 

by F 7 

B 9 = X ® F 9 = (b 0 9 , bn 9 , ... , bn 9 , bj 9 , b i+1 9 , ....) result of input signal X filtered 

by F 9 



B = X ® Fm = (b Q M , br, ... , bi.r, b-r, b i+ r, ....) result of input signal X 
filtered by F M 

An alternate way to empirically represent these filtered output signal samples is as 
follows: 

bi 3 = xm + 2xj + Xj+i 

bj 5 = Xi_2 + 2xi_i + 3Xi + 2x i+1 + x i+2 

b 7 = x,. 3 + 2x h2 + 3x,-i + 4xi + 3x i+1 + 2x j+2 + x i+3 
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bj = Xj_4 + 2Xf_3 + 3Xi_2 + 4xi-i + 5xj + 4x i+ i + 3x i+2 + 2x i+3 + x i+4 

Likewise, by introducing what is referred to, in this context, as state variables, the 
above expressions may be re-expressed as follows: 



bj 3 = x t + Sj 3 , where s 3 = xm + Xj + x i+ i 



b,- 5 = b 3 + Sj 5 , where Sj 5 = Xj_ 2 + x,_i + x* + xm + x i+ 2 



b 7 = bi 5 + Sj 7 , where Sj 7 = x^ + Xj_ 2 + x M + x s + x i+ i + x l+2 + x i+3 



bj 9 = bj 7 + Si 9 , where s 9 = Xj_ 4 + Xj- 3 + Xj_ 2 + xm + x s + x i+ i + x j+2 + x l+3 + x i+4 



Hence, the desired pyramid filter may be expressed as follows: 



B 3 = X + S 3 , where S 3 = (s 0 3 , si 3 , s 2 3 , . . . , s M 3 , s 3 , s i+1 3 , . . .) 



B 5 = B'+S 5 , where S 3 = (s 0 b ,si , s 2 5 , s M 5 , s, , s j+ i , ...) 



B 7 = B 5 + S 7 , where S 7 = (s 0 7 , s/ , s 2 7 , .. . , s M 7 , s 7 , s i+1 7 , .) 



B 9 = B 7 +S 9 ,whereS 9 = (So 9 ,s 1 9 J s 2 9 5 s M 9 , s 9 , s !+1 9 , ...) 
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A study of FIG. 2 illustrates that the computed output signal streams, B 3 , B 5 , B 7 , 
B 9j etc. of the pyramid filters shown in FIG. 2 are produced by the embodiment 
illustrated. 



The previous discussion of pyramid filters occurs in the context of one- 
dimensional filtering; however, due at least in part to the symmetric nature of such 
filters, it is possible to implement pyramid two-dimensional filtering instead of computing 
in a row-wise and column-wise one-dimensional fashion that employs extra 
computational steps. If we represent the one-dimensional k -tap pyramid filter as 



12 3- — - 3 2 1 



, the corresponding two dimensional pyramid 



filter may be derived as shown in FIG. 6. In FIG. 7, we have shown the two- 
dimensional pyramid filter kernel for k=9. Assuming a two-dimensional input signal, e.g., 
signal samples, having the form shown in FIG. 5, FIG, 4 is a table illustrating a matrix 
that may result, here a two-dimensional filtered signal sample output matrix, P kxk , in 
which the two dimensional input signal sample matrix is filtered using two-dimensional 
pyramid filter kernel , . 



The matrix shown in FIG. 8 may result from applying a one-dimensional k-tap 
pyramid filter in every row of the two-dimensional input signal sample matrix and the 
matrix shown in FIG. 9 may result from applying a one-dimensional k-tap pyramid filter 
in every column of the two-dimensional input signal sample matrix. The matrix in FIG, 4 
may result from applying the two-dimensional (k x k) tap filter to the two dimensional 
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input signal sample matrix or, alternatively, it may result from applying the one- 
dimensional k-tap pyramid filter row-wise and then followed by column-wise. Applying 
this approach to generate filtered signal samples outputs P , P , and P , produces 
the following relationships: 



P =S +25 + S 

P-f =S i -ij-x+ 2s i-xj + *MJ + i + 2s i,M +4s U +2s >,M +S MJ-l +2s MJ + Wi 



Generating filtered signal samples outputs P 5x1 , P 1x5 , and P 5x5 , produces the 
following relationships: 



P U = S i-2J + 2s i-U + 3S U + 2S MJ + S MJ 

P U = S U-2 + 2s U-i + 3s u + 2s u^ + S U+2 
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P S =( s i-2j-2 +2 h-2j-x +3*,_ 2J + 2*,_ 2 J+1 +* l _2j +2 ) + (2* ( - u -2 + 4 V U -1 + 6*,_, j + 4s w J+I + 2s w j. 
(3*ij_ 2 +6s u _, +9s (J +6s u+1 +35 <J+2 ) + (2s WJ _ 2 +4s i+u _, +6s (+1J + 45 i+u+1 + 2* i+1J+2 ) + 

( s mj-2 + 2s i+2j~\ + 3s j+2j . + 2s ;+2j . +1 +*,- +2 j +2 ) 



Likewise, generating filtered signal samples outputs P , P , and P , produces 
the following relationships: 

P if = S l-2J + 2s t-U + 3s l-lJ + 4S U + 3g MJ + 2S i+2J + S MJ 
P U = S U-3 + 2s U-2 + 3S i,M + 4S U + 3s U+l + 2S U + 2 + S lJ+3 

p if = ( 5 w.y-3 + 2s i-3j-2 +35,_3j-, +45,_ 3J +3s,_ 3J+1 +2s < _3 >;+2 +5 i _ 3J+3 ) + 

-2,/+2 + 2S i-2,j+3 ) + 

(3s MJ _3 +6j W j_ 2 +95 mj _, +12j wj +9s My . +1 +6s WJ+2 + 3s,._ u+3 ) + 
(4* (J _3 +8*,j-2 +12s u-i +165 /j +12s u + i +8 *u +2 +4* u+3 ) + 

Q S i+\J-3 + ^r+U-2 "^^^i+lj-l + ^ 5 /+lJ 

+ 4s i + ij-2 + 65 i+2J .i + 8s i+2J + 6s l+2J+1 + 4s i+2J+2 + 2s i+2J+3 ) + 

( S i+3J-3 +2S i+3,j-2 + 3s i+3J~i + 4s i+3J + 3*,- + 3j+i + 2S /+3 y +2 +^,+3^+3) 



Furthermore, generating filtered signal samples outputs P , P , and P , produces 
the following relationships: 
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P'f = s,^ + 2s tJ _ 3 + 3s hJ _ 2 + 4s, + 5s, j + 4s 1J+l + 3s iJ+2 + 2s I J+3 + s, J+A 

Pff = (s,._ 4/ ._ 4 + 2s,._ 4> ,_ 3 + 3s,._ 4 ,_ 2 +4s,_ 4J _, + 5s ; _ 4J + 4v 4J+1 +3s ; _ 4J+2 + 2s ; _ 4J+3 + j / _ 4J+4 ) + 

+ 2*1-3 J + 4> + 

(3s,_2j_4 +6*,._ 2J _3 + 9s,_ 2JL . 2 +125 / ._ 2J _ ] +15s,_ 2J +12s # _ 2J+I + 9s,_ 2 J+2 + 6s,._ 2 J+3 + 3s,._ 2J+4 ) + 

(4s HJ _ 4 +8Vu-j +12j,_ U -2 + 16 *mj-i +20j mj + 16 Vi,, + i + 12s,- U+2 + 8s i _ u+3 +4s WJ+4 ) + 
(5s. ; _ 4 + 10s f ,._ 3 +15s u _ 2 + 20*,^ +255, j +2Qs ij+x + l5s iJ+2 +\0s ij+3 +5s iJ+4 ) + 
(^m,M +8 Vu-3 + 12*» + ,j-2 +16* <+u _, +205,. +1 . +165 /+u+1 +12s i+1J+2 +to (+u+3 +4j i+u+4 ) + 

(*,* + 4 j_4 + 2s i+4 j_ 3 + 35 f+4 j_ 2 + 45 i+4 j_j -f 5s M j + 4 k y r - +4 y +1 -f 3^ f+4 + 2s i+4J+3 + s i+4 j+ 4 ) 



Mathematical manipulation may be employed to produce the following: 



p9x9 
ij 



- (P™ + 2P% + /> |f + IP™ + P/X- ) 



[1] 

P 7 * 7 ^-/'P 9 * 1 



P>f+2(P>f+Plf)-4s u 
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Equation [1] above illustrates that a direct two-dimensional pyramid filter 
architecture of order 2N-1 , in this case where N is five, may potentially be implemented 
using either four two-dimensional pyramid filters of order [2(N-1)-1], that is seven, or 
one two-dimensional pyramid filter of order [2(N-1)-1] using four signal sample matrices 
^I^-^^I^+^Pj^-i,^!^^ and ten one-dimensional pyramid filters of order 2N-1, here 
nine, the filters being row-wise and column-wise, in this example. It also employs one 
two-dimensional pyramid filter of order [2(N-2)-1], that is five here to produce P*J 5 and 
two one-dimensional pyramid filters of order [2(N-2)-1], that is five here, to produce two 
signal sample matrices P** 5 ,p}j 5 , in this example. FIG. 3 is a schematic diagram 

illustrating such an embodiment, although, of course, the claimed subject matter is not 
limited in scope to this particular implementation or embodiment. For example, the 
output signal samples corresponding to those produced by four two-dimensional 
pyramid filters of order 2(N-t)-1 , here order seven where N is five, and also the output 
signal samples produced by two-dimensional pyramid filter of order 2(N-2)-1, here order 
five, may not necessarily be produced by two-dimensional pyramid filters. As just one 
example, these output signals may be produced using one-dimensional pyramid filters. 
One such filter is shown in FIG. 2, although, again, additional approaches to producing 
the output signals for the architecture shown in FIG. 3 may also be employed. 

FIG. 3 illustrates an integrated circuit (IC), 300, although, of course, alternative 
embodiments may not necessarily be implemented on a single integrated circuit chip. 
IC 300 includes a two-dimensional pyramid filter architecture of an order 2N-1, where N 
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is a positive integer greater than four, here five, in operation, is capable of producing, 
on respective clock cycles, at least the following. Pyramid filtered output signals are 
produced corresponding to output signals produced by ten one-dimensional pyramid 
filters of order 2N-1 , again, nine in this example where N is five, 330, 332, 334, 340, 
342, 344, 350, 352, 354 and 360 in FIG. 3. Pyramid filtered output signals are also 
produced corresponding to output signals produced either by four two-dimensional 
pyramid filters or one two-dimensional pyramid of order [2(N-1) - 1] or seven here, 

where N is five, using signal sample matrices P t 7 Xj-\ •> •> > P ^Z^ - These 
output signals are summed by adder 310 in FIG. 3. Pyramid filtered output signals are 
also produced corresponding to output signals produced by one two-dimensional 
pyramid filter of order [2(N-2) - 1] or five here, where N is five, using signal sample 
matrix P^/ 5 and two one-dimensional pyramid filter of order 2(N-2)-1 using signal 

sample matrices Pff ,Plf . These three output signals P^ v \ P-f^P-* 5 and the 

input signal s u are summed by adder 390 in FIG. 3. Likewise, the respective output 

signals in this two dimensional pyramid filter architecture implementation, in the 
implementation in FIG. 3, for example, the output signals of 330, 332, 334, 340, 342, 
344, 350, 352, 354 and 360, are summed on respective clock cycles of the two 
dimensional pyramid filter architecture, by adders 370 and 375 in FIG. 3. Adder 380 
sums the output signals of 310, 370, 375 and 390. Of course, FIG. 3 is just one 
possible example of an implementation and the claimed subject matter is not limited in 
scope to this or to another particular implementation. 

For example, N is not limited to five. Likewise, the pyramid filtered output signals 
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that correspond to output signals produced by a two-dimensional pyramid filter are not 
limited to being implemented by one-dimensional pyramid filters or to two-dimensional 
pyramid filters. Likewise, as previously indicated, if one-dimensional filters are 
employed, then the filters are not limited to the implementation approach described in 
aforementioned U. S, Patent Application Serial No. 09/754,684, titled "Multiplierless 
Pyramid Filter," filed January 3, 2001, by Tinku Acharya, or in aforementioned U. S. 
Patent Application Serial No., titled "Pyramid Filter," (attorney docket 042390.P11211), 
filed on March 28, 2001 , by Tinku Acharya. For example, one-dimensional pyramid 
filters other than multiplierless pyramid filters may be employed. Likewise, depending 
on the implementation, different numbers of such pyramid filters and different orders of 
such pyramid filters may be employed. For example, the output signals may be 
combined or processed in a way to produce pyramid filtered output signals 
corresponding to pyramid filters of a different number, dimension, or order. 

It will, of course, be understood that, although particular embodiments have just 
been described, the claimed subject matter is not limited in scope to a particular 
embodiment or implementation. For example, one embodiment may be in hardware, 
whereas another embodiment may be in software. Likewise, an embodiment may be in 
firmware, or any combination of hardware, software, or firmware, for example. 
Likewise, although the claimed subject matter is not limited in scope in this respect, one 
embodiment may comprise an article, such as a storage medium. Such a storage 
medium, such as, for example, a CD-ROM, or a disk, may have stored thereon 
instructions, which when executed by a system, such as a computer system or platform, 
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or an imaging system, for example, may result in an embodiment of a method in 
accordance with the claimed subject matter being executed, such as an embodiment of 
a method of filtering or processing an image or video, for example, as previously 
described. For example, an image processing platform or an imaging processing 
system may include an image processing unit, a video or image input/output device 
and/or memory. 

While certain features of the claimed subject matter have been illustrated and 
described herein, many modifications, substitutions, changes and equivalents will now 
occur to those skilled in the art. It is r therefore, to be understood that the appended 
claims are intended to cover all such modifications and changes as fall within the true 
spirit of the claimed subject matter. 
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